Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Html out #531

Closed
wants to merge 223 commits into from
Closed

Html out #531

wants to merge 223 commits into from

Conversation

shmfr
Copy link
Collaborator

@shmfr shmfr commented Jun 14, 2023

No description provided.

…ubeSet) holding all connections including src_peers, dst_peers and protocols

Signed-off-by: Tanya <[email protected]>
Added optimized_denied_ingress_props
and optimized_denied_egress_props (in addition to allowed ones).
Improved non_captured_conns computation

Signed-off-by: Tanya <[email protected]>
Signed-off-by: Tanya <[email protected]>
…_HC_set

# Conflicts:
#	nca/CoreDS/CanonicalHyperCubeSet.py
Signed-off-by: Tanya <[email protected]>

Signed-off-by: Tanya <[email protected]>
Signed-off-by: Tanya <[email protected]>

Signed-off-by: Tanya <[email protected]>
Fixed protocol handling: 0 is a valid protocol number (HOPOPT).
Allowing any protocol in the range [0...255], though ProtocolNameResolver does not contain names of all of the possible 256 protocols.
Fixed handling non-captured peers in K8S (cannot be handled as denied).

Signed-off-by: Tanya <[email protected]>
…r dst_peers dimensions is all values).

Added optimization for fw_rules_map - join different entries having the same values (fw_rules).

Signed-off-by: Tanya <[email protected]>
1. To represent No connections, do not build TcpLikeProperties (TcpLikeProperties with no dimensions represent All connections).
2. Support subsets in query in optimized solution.
3. For comparison of optimized solution to the original one, add connections from peers to themselves.
4. More accurate comparison for 'dot' connectivity queries.
5. Generalized convert_named_ports (to not assume dimensions order)
6. Handling the possibility when projection on one dimension is empty.

Signed-off-by: Tanya <[email protected]>
Properly handling HostEPs in optimized TcpLikeProperties.

Signed-off-by: Tanya <[email protected]>
…perties)

Fixed handling of non-captured pods in optimized solution.
Added using True/False HC_set (make_all_properties()/make_empty_properties())

Signed-off-by: Tanya <[email protected]>
'false' - only original run
'true' - only optimized run
'debug'- both runs and comparison of their results.

Printing parsing time, queries time and total run time.

Signed-off-by: Tanya <[email protected]>
Workaround for the bug in HC set: using mutual contained_in, instead of ==

Signed-off-by: Tanya <[email protected]>
Signed-off-by: Tanya <[email protected]>
…for -opt=debug option)

Added more debug prints.
Better handling of peer_set copying in TcpLikeProperties.

Signed-off-by: Tanya <[email protected]>
Further optimization - calculating ref_ip_blocks only in non-optimized run.

Signed-off-by: Tanya <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr shmfr marked this pull request as ready for review June 18, 2023 12:33
@shmfr shmfr requested a review from adisos June 18, 2023 12:33
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Copy link
Member

@adisos adisos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a few initial comments.

  • add basic tests with output of svg file

@@ -321,7 +325,7 @@ def nca_main(argv=None):
help='A list of labels to subset the query by')
parser.add_argument('--ghe_token', '--gh_token', type=str, help='A valid token to access a GitHub repository')
parser.add_argument('--output_format', '-o', type=str,
help='Output format specification (txt, txt_no_fw_rules, csv, md, dot, jpg or yaml). '
help='Output format specification (txt, txt_no_fw_rules, csv, md, dot, jpg, html or yaml). '
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add the new html format in the relevant readme files, possibly also include some explanation (and also assumption that graphviz is installed?)

@@ -327,6 +393,218 @@ def create_output(self, elements_relations):
self._highlight_tag(related_tag, related_tag_info.t_class)
self._save_tag_file(tag_soup, tag_info)

HTML_TEMPLATE = '''
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this template be saved in a separate file?

shmfr added 11 commits June 26, 2023 18:33
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Signed-off-by: Shmulik Froimovich <[email protected]>
Base automatically changed from explainability_with_HCS to master July 6, 2023 06:40
@@ -59,13 +60,13 @@ class ElementRelations:
highlights: set = field(default_factory=set)
explanation: list = field(default_factory=set)

def __init__(self, svg_file_name, output_directory):
def __init__(self, svg_file_name, output_directory, expl_xml=None):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add the new arg to documentation.

@@ -62,6 +64,23 @@ def print_query_output(self, output, supported_output_formats=None):
print(f'Command {dot_cmd_string}\n did not create {path}\n', file=sys.stderr)
if os.path.isfile(tmp_dot_file):
os.remove(tmp_dot_file)
elif self.outputFormat == 'html':
Copy link
Member

@adisos adisos Jul 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trying to run the following command from cli:
python -m nca --connectivity -r tests/fw_rules_tests/podlist/ns_list.json -r tests/fw_rules_tests/podlist/pods_list_4.json -r tests/fw_rules_tests/policies/test2-networkpolicy.yaml -o html -f test2.html

but getting:

Failed to create a svg file: test2.html
'NoneType' object is not iterable
Command dot test2.html.nca_tmp.dot -Tsvg -otest2.html.nca_tmp.svg
 did not create test2.html
  • I do have dot installed on my env
  • we should avoid the error 'NoneType' object is not iterable even if dot is not installed

if alphabet_str:
fqdn_regex = r"(?=.{1,254}$)[A-Za-z0-9]([-A-Za-z0-9]*[A-Za-z0-9])?(\.[A-Za-z0-9]([-A-Za-z0-9]*[A-Za-z0-9])?)*[.]?"
fqdn_regex = "^((?!-)[A-Za-z0-9-]+(?<!-).)+[A-Za-z0-9.]+"

Check failure

Code scanning / CodeQL

Inefficient regular expression High

This part of the regular expression may cause exponential backtracking on strings containing many repetitions of '--'.
@adisos adisos closed this Jan 15, 2024
@shmfr shmfr linked an issue Jan 15, 2024 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

connect interactive map to NCA
4 participants